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(54) Image sensing device, image processing apparatus and method, and memory medium 



(57) An image sensing device of this invention cor- 
rects sensitivity nonunifbrmity of an image sensing ele- 
ment. This image sensing device has an image sensing 
element (5) having a two-dimensional matrix of pixels, a 
first multiplier circuit (18) for multiplying the output from 
the image sensing element (5) output via an A/D con- 
verter (7) by horizontal linear correction data, a second 
multiplier circuit (19) for multiplying the product of the 



first multiplier circuit by vertical linear correction data, a 
selector (16) for selecting horizontal linear correction 
data, and a selector (17) for selecting vertical linear cor- 
rection data. The device switches the selectors (16, 17) 
in accordance with the position (horizontal and vertical 
addresses) of the pixel of interest. 
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Description 

FIELD OF THE INVENTION 

[0001] The present invention relates to an image 
sensing device, an image processing apparatus and 
method, and a memory medium. 

BACKGROUND OF THE INVENTION 

[0002] Fig. 16 is a schematic block diagram show- 
ing the arrangement of a conventional digital still cam- 
era. In this digital camera, an overall control circuit 80 
detects changes in state of operation switches (main 
switch, release switch, and the like) 91, and starts 
power supply to other blocks. 

[0003] An object image within the photographing 
frame range is formed on the image sensing surface of 
an image sensing unit 82 via a main photographing opti- 
cal system 81. The image sensing unit 82 converts this 
image into an analog electrical signal, and supplies it to 
an A/D converter 83 in turn in units of pixels. The A/D 
converter 83 converts the analog electrical signal into a 
digital signal, and supplies the digital signal to a process 
circuit 84. 

[0004] The process circuit 84 generates R, G. and 
B image signals on the basis of the input data. In the 
state before photographing, these image signals are 
periodically transferred to a video memory 89 via a 
memory controller 85 in units of frames, thus displaying 
an image on a display unit 90. In this manner, a view- 
finder display is implemented. 

[0005] On the other hand, when the photographer 
operates one of operation switches 91 to instruct execu- 
tion of photographing, pixel data for one frame output 
from the process circuit 84 are stored in a frame mem- 
ory 86 in accordance with a control signal output from 
the overall control circuit 80. The data in the frame 
memory 86 are compressed by the memory controller 
85 and a work memory 87 on the basis of a predeter- 
mined compression format, and the compression result 
is stored in an external memory (e.g., a nonvolatile 
memory such as a flash memory or the like) 88. 
[0006] When the photographer reviews photo- 
graphed images, the compressed data stored in the 
external memory 88 are read out, and are expanded by 
the memory controller 85 to be converted into normal 
data in units of photographing pixels. The conversion 
result is transferred to the video memory 89, and an 
image is displayed on the display unit 90. 
[0007] In such digital camera, microlenses 31 
shown in Figs. 2A and 2B are provided to an image 
sensing element of the image sensing unit 32 in units of 
photosensitive pixels so as to improve optical sensitivity 
in units of pixels. In Figs. 2A and 2B, reference numeral 
30 denotes a photographing lens (main photographing 
optical system) as a field lens; and 32, each photosen- 
sitive section (light- receiving section) of the image sens- 



ing element. 

[0008] Since the microlenses 31 are provided in 
units of photosensitive pixels of the image sensing ele- 
ment, even when the image sensing element has a nar- 
5 row effective sensitivity range, marginal light can be 
effectively focused on photosensitive pixels. 
[0009] When light rays that have passed through 
the photographing lens 30 strike the image sensing ele- 
ment in a direction nearly parallel to the optical axis, as 
10 shown in Fig. 2A, incoming light rays focus on the pho- 
tosensitive section 31 without posing any serious prob- 
lem. However, when light rays obliquely enter the 
photographing lens 30, as shown in Fig. 2A, only some 
incoming light rays become incident on the photosensi- 
15 tive section 31 in an area (peripheral portion of the 
image sensing element) separated from the optical axis 
of the photographing lens 30. 

[0010] Such light amount drop is normally called 
white shading. This phenomenon becomes conspicu- 

20 ous as the pixel position on the image sensing element 
separates farther away from the optical axis of the pho- 
tographing lens, and also as the focal length of the pho- 
tographing lens 30 decreases (in practice, as the 
distance from the image sensing element to the pupil 

25 position of the photographing lens becomes shorter). 
[0011] Figs. 3A and 3B are graphs showing 
changes in white shading when the stop of the photo- 
graphing lens 30 has changed. Figs. 3A and 3B respec- 
tively show the relationship between the image height 

30 and relative sensitivity on the image sensing element in 
a full-aperture state (Fig. 3A), and a stop-down state 
(Fig. 3B). In the full-aperture state of the stop, the rela- 
tive sensitivity drops largely compared to the central 
portion with increasing height of an image formed on 

35 the image sensing element, since many light compo- 
nents that obliquely enter the photographing optical sys- 
tem are included, as shown in Fig. 2B. 
[0012] On the other hand, when the stop is stopped 
down, the relative sensitivity changes little even when 

40 the image height becomes larger, since light compo- 
nents that obliquely enter the photographing lens 30 are 
suppressed by the stop effect. 

[0013] As a method of correcting white shading pro- 
duced by a combination of the photographing lens 30 

45 and microlenses on the image sensing element, a 
method disclosed in Japanese Patent Laid-Open No. 9- 
130603 is known. In this method, assuming that each 
pixel position on the image sensing element is 
expressed by a pointer indicated by a horizontal direc- 

so tion X and vertical direction Y, shading correction data 
H(x) for one horizontal line, and shading correction data 
V(y) for one vertical line are used. The shading correc- 
tion data for one horizontal line and one vertical line 
assume larger values as the pixel position approaches 

55 an end (periphery). A pixel S(i, j) on the image sensing 
element is multiplied by horizontal correction data H(i) 
and vertical correction data V(j) as per: 
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S(i,j)xH(j)xV(j)-»S' 

In this way, sensitivity drop in the peripheral portion of 
the image sensing element due to shading is apparently 
prevented. 5 
[0014] When a digital camera is constructed using 
the same optical system (e.g., exchangeable lens sys- 
tem) as that of a conventional single-lens reflex silver 
halide camera, an image sensing element having an 
image sensing area considerably larger than that of a 10 
normal image sensing element is required. 
[0015] However, an image sensing element having 
such a large image sensing area often suffers a problem 
of sensitivity nonuniformity of color filters and the like, 
which is produced in the manufacture process. In view 15 
of the whole image sensing element, only a local area 
suffers a decrease or increase in sensitivity, which can- 
not be corrected by simply setting higher gains toward 
the periphery. 

[0016] In Fig. 4, 401 indicates an example of the 20 
sensitivity distribution of the image sensing element, in 
which the central sensitivity is higher than the peripheral 
one. Such sensitivity nonuniformity can be corrected to 
some extent using the aforementioned conventional 
method. 25 
[0017] For example, 402 in Fig. 4 expresses a func- 
tion of horizontal sensitivity correction data H(i), which 
has smaller correction coefficient values at the central 
portion and larger ones at the periphery. On the other 
hand, 403 in Fig. 4 expresses a function of vertical sen- 30 
sitivity correction data Vfl), which has smaller correction 
coefficient values at the central portion and larger ones 
at the periphery. 

[0018] Therefore, correction can be done using 
such functions by calculating a value S(i, j) at each pixel 35 
point as per: 

S(i, j)xH(i)xV(j)->S' (I, j) 

[0019] However, when local sensitivity nonuniform- 40 
ity is present in a plurality. of areas on the frame, as indi- 
cated by 501 in Fig. 5, it cannot be perfectly removed by 
the above-mentioned correction method that simply 
uses shading correction data H(x) for one horizontal line 
and shading correction data V(y) for one vertical line. 45 
[0020] On the other hand, in the device structure of 
an image sensing element indicated by 601 in Fig. 6, 
since aluminum interconnects and the like run to sand- 
wich, e.g., two sensitivity areas, the level of incoming 
light to one (G sensitivity area) of two sensitivity areas so 
lowers in areas on the left side in Fig. 6, while one (R 
sensitivity area) of two sensitivity areas lowers in areas 
on the right side in Fig. 6. Therefore, the two sensitivity 
areas (G and R) have different sensitivity levels depend- 
ing on the right and left areas of the frame of the image 55 
sensing element, as indicated by 602 in Fig. 6. 
[0021] Likewise, in neighboring rows (rows in which 
G and R alternately appear), the sensitivity distribution 



changes, as indicated by 603 in Fig. 6. That is, in this 
case, since a Bayer matrix is assumed as a matrix of 
color filters, G sensitivity lowers to the right unlike the 
characteristics indicated by 602. 
[0022] When colors are formed by combining R, G, 
and B with such characteristics, different color tones are 
obtained on the right, left, up, and down positions on the 
frame. Such phenomenon is generally called color 
shading, which is also produced by the aforementioned 
combination of the photographing lens and micro- 
lenses, in addition to the device structure. 
[0023] Against such phenomenon, color unbal- 
ances cannot be sufficiently corrected by only combin- 
ing the conventional linear correction data sequences 
H(i) and V(j). 

SUMMARY OF THE INVENTION 

[0024] The present invention has been made in 
consideration of the above situation, and has as its 
object to correct, e.g., sensitivity nonuniformity of an 
image sensing element. 

[0025] More specifically, it is an object of the 
present invention to correct, e.g., local sensitivity nonu- 
niformity of an image sensing element resulting from 
variations or the like of color filters provided to the image 
sensing element in the manufacture process. 
[0026] It is another object of the present invention to 
correct sensitivity nonuniformity (e.g., color shading, 
white shading) of an image sensing element resulting 
from a combination of the device structure of the image 
sensing element and a photographing optical system. 
[0027] It is still another object of the present inven- 
tion to correct sensitivity nonuniformity (e.g., color shad- 
ing, white shading) of an image sensing element 
resulting from a combination of the device structure of 
the image sensing element and a photographing optical 
system independently from a change in state of the pho- 
tographing optical system. 

[0028] Other objects of the present invention will be 
described in the embodiments of the present invention. 
[0029] An image sensing device according to the 
first aspect of the present invention comprises an image 
sensing element having a two-dimensional matrix of 
pixels, first multiplier means for multiplying a row of an 
image sensed by the image sensing element by hori- 
zontal linear correction data, second multiplier means 
for multiplying a column of the image by vertical linear 
correction data, and change means for changing at 
least one of the horizontal and vertical linear correction 
data in accordance with a position of a pixel of interest 
in the image, and the first and second multiplier means 
correct a value of each pixel of the image. 
[0030] In the image sensing device according to the 
first aspect of the present invention, for example, the 
image is preferably segmented into a plurality of partial 
areas, and the change means changes at least one of 
the horizontal and vertical linear correction data in 
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accordance with the partial area to which the pixel of 
interest belongs. 

[0031] In the image sensing device according to the 
first aspect of the present invention, for example, the 
image is preferably segmented into a plurality of partial 5 
areas, and the change means changes both the hori- 
zontal and vertical linear correction data in accordance 
with the partial area to which the pixel of interest 
belongs. 

[0032] In the image sensing device according to the w 
first aspect of the present invention, for example, the 
change means preferably has two different types of hor- 
izontal linear correction data, alternately switches the 
horizontal linear correction data in units of rows, and 
supplies the selected data to multiplication by the first 15 
multiplier means. 

[0033] In the image sensing device according to the 
first aspect of the present invention, for example, the 
change means preferably has two different types of ver- 
tical linear correction data, alternately switches the ver- 20 
tical linear correction data in units of columns, and 
supplies the selected data to multiplication by the sec- 
ond multiplier means. 

[0034] In the image sensing device according to the 
first aspect of the present invention, for example, the 25 
image sensing element preferably includes color filters 
in a Bayer matrix. 

[0035] In the image sensing device according to the 
first aspect of the present invention, for example, the 
image is preferably segmented into a plurality of groups 30 
each of which is defined by an area as a two-dimen- 
sional matrix of a plurality of pixels, the device further 
comprises third multiplier means for multiplying each 
group by two-dimensional correction data given by a 
row and column, and the change means includes 35 
means for changing the two-dimensional correction 
data in accordance with the group to which the pixel of 
interest in the image belongs. 

[0036] In the image sensing device according to the 
first aspect of the present invention, for example, the 40 
change means preferably includes means for changing 
at least one of the horizontal and vertical linear correc- 
tion data in accordance with a state of a photographing 
optical system. 

[0037] In the image sensing device according to the 45 
first aspect of the present invention, for example, the 
change means preferably includes means for changing 
at least one of the horizontal and vertical linear correc- 
tion data in accordance with one of a focal length, field 
angle, and aperture of a photographing optical system 50 
or a combination thereof. 

[0038] An image processing apparatus according to 
the second aspect of the present invention is an image 
processing apparatus for processing an image supplied 
from an image sensing element having a two-dimen- 55 
sional matrix of pixels, comprising first multiplier means 
for multiplying a row of an image sensed by the image 
sensing element by horizontal linear correction data, 
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second multiplier means for multiplying a column of the 
image by vertical linear correction data, and change 
means for changing at least one of the horizontal and 
vertical linear correction data in accordance with a posi- 
tion of a pixel of interest in the image, wherein the first 
and second multiplier means correct a value of each 
pixel of the image. 

[0039] An image processing method according to 
the third aspect of the present invention comprises the 
first multiplier step of multiplying a row of an image 
sensed by an image sensing element having a two- 
dimensional matrix of pixels by horizontal linear correc- 
tion data, the second multiplier step of multiplying a col- 
umn of the image by vertical linear correction data, and 
the change step of changing at least one of the horizon- 
tal and vertical linear correction data in accordance with 
a position of a pixel of interest in the image, and a value 
of each pixel of the image is corrected in the first and 
second multiplier steps. 

[0040] A memory medium according to the fourth 
aspect of the present invention is a memory medium 
storing an image processing program, the program 
including the first multiplier step of multiplying a row of 
an image sensed by an image sensing element having 
a two-dimensional matrix of pixels by horizontal linear 
correction data, the second multiplier step of multiplying 
a column of the image by vertical linear correction data, 
and the change step of changing at least one of the hor- 
izontal and vertical linear correction data in accordance 
with a position of a pixel of interest in the image, 
wherein a value of each pixel of the image is corrected 
in the first and second multiplier steps. 
[0041] Other features and advantages of the 
present invention will be apparent from the following 
description taken in conjunction with the accompanying 
drawings, in which like reference characters designate 
the same or similar parts throughout the figures thereof. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0042] The accompanying drawings, which are 
incorporated in and constitute a part of the specification, 
illustrate embodiments of the invention and, together 
with the description, serve to explain the principles of 
the invention. 

Fig. 1 is a block diagram showing the arrangement 
of a camera (image sensing device) according to 
the first and second embodiments of the present 
invention; 

Figs. 2A and 2B show a combination of an image 
sensing element and photographing optical system; 
Figs. 3A and 3B are graphs for explaining white 
shading; 

Fig. 4 shows an example of the sensitivity distribu- 
tion and correction of the image sensing element; 
Fig. 5 is a view for explaining the method of correct- 
ing the sensitivity of the image sensing element 
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according to the first embodiment of the present 
invention; 

Fig. 6 is a view for explaining the device structure of 
the image sensing element and a phenomenon 
resulting from that structure; 
Fig. 7 is a view for explaining the color filter matrix 
of the image sensing element and a color interpola- 
tion method; 

Fig. 8 is a flow chart for explaining initial setup oper- 
ation in the first and second embodiments of the 
present invention; 

Fig. 9 is a flow chart for explaining correction oper- 
ation according to the first embodiment of the 
present invention; 

Fig. 1 0 is a flow chart for explaining correction oper- 
ation according to the second embodiment of the 
present invention; 

Fig. 1 1 is a view for explaining the concept of sensi- 
tivity correction according to the second embodi- 
ment of the present invention; 
Fig. 12 is a block diagram showing the arrangement 
of a camera (image sensing device) according to 
the third embodiment of the present invention; 
Fig. 1 3 is a view for explaining the concept of sensi- 
tivity correction according to the third embodiment 
of the present invention; 

Fig. 14 is a flow chart for explaining correction oper- 
ation according to the third embodiment of the 
present invention; 

Figs. 15A and 15B are views for explaining the sen- 
sitivity correction method according to the third 
embodiment of the present invention; and 
Fig. 16 is a block diagram for explaining the 
arrangement of a conventional camera. 

DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

[First Embodiment] 

[0043] Fig. 1 is a block diagram showing the 
arrangement of a camera (image sensing device) 
according to a preferred embodiment of the present 
invention. Referring to Fig. 1, reference numeral 1 
denotes an overall control CPU for controlling the over- 
all camera. Reference numeral 2 denotes a focal length 
detector for detecting the focal length of a photograph- 
ing optical system of the camera (or the pupil position of 
the photographing optical system). The detector 2 out- 
puts encoded information corresponding to the focal 
length and supplies it to the CPU 1. Reference numeral 
3 denotes an aperture detector for detecting the state of 
an aperture in the photographing optical system. The 
detector 3 outputs encoded information corresponding 
to the aperture state, and supplies it to the CPU 1. 
[0044] Reference numeral 4 denotes a main photo- 
graphing optical system (photographing lens) of the 
camera. Reference numeral 5 denotes an image sens- 



ing element for converting optical information as an 
object image formed by the main photographing optical 
system 4 into an electrical signal, and outputting the 
electrical signal. The image sensing element 5 com- 
5 prises, e.g., a charge transfer element such as a CCD or 
the like. 

[0045] Charges accumulated by the image sensing 
element 5 within a predetermined period of time are 
read in turn in units of pixels, and are supplied to a 

10 CDS/AGC circuit 6. The circuit 6 reduces noise compo- 
nents such as reset noise and the like produced by the 
image sensing element itself, amplifies the charge sig- 
nal to an appropriate level, and supplies the signal to an 
A/D converter 7. The A/D converter 7 converts object 

15 luminance information corresponding to the charge 
amount into digital data. Since optical color filters for 
generating, e.g., R, G, and B color signals or the like are 
adhered onto the image sensing element 5, the output 
signals from the image sensing element 5 alternately 

20 represent the respective colors. 

[0046] Reference numeral 8 denotes a driver circuit 
for actually driving the image sensing element 6. The 
driver circuit 8 supplies driving pulses to the image 
sensing element 5 at given periods on the basis of a tim- 

25 ing signal supplied from a timing generator circuit 9. 
[0047] The timing generator circuit 9 generates hor- 
izontal and vertical sync signals HD and VD and sup- 
plies them to an address generation circuit 10. The 
address generation circuit 10 generates address sig- 

30 nals to be supplied to memories 12 to 15 connected to 
the output side of the circuit 10 on the basis of the hori- 
zontal and vertical sync signals HD and VD. 
[0048] Reference numerals 12 to 15 denote memo- 
ries for storing data used in sensitivity correction of the 

35 image sensing element. More specifically, reference 
numeral 12 denotes an H memory H1 for storing hori- 
zontal first correction data; 13, an H memory H2 for stor- 
ing horizontal second correction data; 14, a V memory 
V1 for storing vertical first correction data; and 15, a V 

40 memory V2 for storing vertical second correction data. 
[0049] Reference numeral 11 denotes an address 
discrimination circuit for outputting a select signal HSEL 
that selects the memory 12 or 13, and a select signal 
VSEL that selects the memory 14 or 15 on the basis of 

45 the signals supplied from the address generation circuit 
10. 

[0050] Reference numerals 16 and 17 denote 
selectors for respectively selecting the outputs from the 
memory 12 or 13 and the memory 14 or 15 on the basis 

so of the select signals HSEL and VSEL supplied from the 
address discrimination circuit 11. 
[0051] Reference numeral 18 denotes a first multi- 
plier circuit (MUL1) for multiplying the output from the 
A/D converter 7 by horizontal correction data obtained 

55 via the selector 16. Reference numeral 19 denotes a 
second multiplier circuit (MUL2) for multiplying the out- 
put from the first multiplier circuit (MUL1) 18 by vertical 
correction data obtained via the selector 17. 
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[0052] The output value from the second multiplier 
circuit (MUL2) 19 is input to, e.g., a process circuit, and 
undergoes dark level correction, y correction, color 
interpolation, and the like. After that, the processed data 
is stored in, e.g., a memory. 5 
[0053] The color interpolation will be explained 
below with reference to Fig. 7. The pixel matrix (color fil- 
ter matrix) of the image sensing element shown in Rg. 
7 has a general Bayer matrix, in which G pixels are 
arranged in a checkerboard pattern, and R/B pixels are 10 
arranged line-sequentially. In case of a single-board 
image sensing element, since all pixels do not have 
RGB information, it is a common practice to generate 
RGB color information corresponding to each pixel posi- 
tion on the image sensing element by interpolation 15 
using 3x3 matrices illustrated at the center of Fig. 7. 
[0054] In Fig. 7, a G interpolation filter is different 
from an R/B interpolation filter. For example, G data at 
position a is generated by respectively multiplying lumi- 
nance data of pixels in area a\ i.e., those of a pixel at 20 
position a and its eight surrounding pixels by coeffi- 
cients of the G interpolation filter. In case of Fig. 7, since 
the coefficient for luminance data at position a corre- 
sponding to a G color filter is 1, and coefficients for 
upper, lower, and right and left neighboring luminance 25 
data are 0.25, but G data at those positions are zero, G 
data is determined by only the output value at position 
a. 

[0055] On the other hand, G data at position b is 
generated by respectively multiplying luminance data of 30 
pixels in area b\ i.e., those of a pixel at position b and its 
eight surrounding pixels by coefficients of the G interpo- 
lation filter. In this case, since G data at position b is 
zero, G data at position b is determined by the average 
value of its upper, lower, right, and left neighboring G 35 
data. 

[0056] Likewise, R/B data for all pixel positions are 
determined using the R/B interpolation filter different 
from the G interpolation filter. In this manner, RGB data 
for all the pixel positions can be generated, as illustrated 40 
at the right end in Fig. 7. 

[0057] The correction process in the camera shown 
in Fig. 1 will be explained below with reference to Figs. 
5, 8, and 9. 

[0058] Fig. 8 is a flow chart showing the process for 45 
initially setting up data of the H memory (H1) 12, H 
memory (H2) 13, V memory (V1) 14, and V memory 
(V2) 15 and the address discrimination circuit 1 1 by the 
overall control CPU 1. 

[0059] In step 24, a linear data sequence H 1 (i) indi- so 
cated by a graph 502 in Fig. 5 is set (stored) in the H 
memory (H1) 12. In step 25, a linear data sequence 
H2(i) indicated by a graph 503 in Fig. 5 is set (stored) in 
the H memory (H2) 1 3. These two data sequences H 1 (i) 
and H2(i) contain different data. These data are deter- 55 
mined in accordance with the characteristics of the 
image sensing element 5 measured during the manu- 
facturing process, and are pre-stored in the internal 
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nonvolatile memory or the like of the overall control CPU 
1. Note that i indicates the horizontal address. In 501 in 
Fig. 5, the left side corresponds to the lower horizontal 
address. 

[0060] Likewise, a linear data sequence V1 (j) indi- 
cated by a graph 504 in Fig. 5 is set (stored) in the V 
memory (V1) 14 in step 26, and a linear data sequence 
V2Q) indicated by a graph 505 in Fig. 5 is set (stored) in 
the V memory (V2) 15 in step 27. These two data 
sequences V1 (j) and V2Q) also contain different data, 
which are determined in accordance with the character- 
istics of the image sensing element 5 measured during 
the manufacturing process, and are pre-stored in the 
internal nonvolatile memory or the like of the overall 
control CPU 1. Note that j indicates the vertical address. 
In 501 in Fig. 5, the upper side corresponds to the lower 
vertical address. 

[0061 ] The position of each pixel of the image sens- 
ing element 5 is specified by horizontal and vertical 
addresses i and j. These horizontal and vertical 
addresses i and j are generated by the address genera- 
tion circuit 10 in accordance with the horizontal and ver- 
tical sync signals HD and VD. 

[0062] In steps 28 and 29, discrimination condition 
values used to discriminate the horizontal and vertical 
addresses generated by the address generation circuit 
10 in the address discrimination circuit 11 in Fig. 1 are 
set. 

[0063] The overall control CPU 1 transfers, as hori- 
zontal address discrimination data, horizontal address 
Ha indicated by 501 in Fig. 5 to the address discrimina- 
tion circuit in step 28, and transfers, as vertical address 
discrimination data, V*p indicated by 501 in Fig. 5 to the 
address discrimination circuit 1 1 in step 29. These two 
data Ha and Vp are determined in accordance with the 
characteristics of the image sensing element 5 meas- 
ured during the manufacturing process, and are pre- 
stored in the internal nonvolatile memory or the like of 
the overall control CPU 1. 

[0064] Switching of correction data upon correcting 
data output from the A/D converter 7 in the first multi- 
plier circuit (MUL1) 18 and second multiplier circuit 
(MUL2) 19 will be explained below with reference to Fig. 
9. Fig. 9 is a flow chart showing the address discrimina- 
tion process in the address discrimination circuit 11. 
[0065] In this example, the first and third areas of 
the image sensing element 5 have central higher sensi- 
tivity portions due to variations of color filters and the 
like in the manufacture, as indicated by 501 in Fig. 5. In 
order to correct such sensitivity distribution, data 
sequences which have smaller correction coefficient 
values in correspondence with a higher filter sensitivity 
portion, and larger correction coefficient values in corre- 
spondence with a lower filter sensitivity portion. 
[0066] In steps 30, 31, and 36 in Fig. 9, one of the 
first to fourth effective areas of the image sensing ele- 
ment 5, to which the position of data (pixel) currently 
output from the A/D converter 7 belongs is determined 



6 



EP 1 067 777 A2 



12 



on the basis of horizontal and vertical addresses i and j. 
[0067] If the vertical address is smaller than vp and 
the horizontal address is smaller than Ha, it is deter- 
mined that the data of interest belongs to the first area 
in 501 of Fig. 5, and the select signals HSEL and VSEL 
as the output signals from the address discrimination 
circuit 1 1 are respectively set at L level in steps 32 and 
33. Correction data H1(i) stored in the H memory (H1) 
12 is supplied to the first multiplier circuit (MUL1) 18 via 
the selector 16, and correction data V1(j) stored in the V 
memory (V1) 14 is supplied to the second multiplier cir- 
cuit (MUL2) 19 via the selector 17. The correction proc- 
ess in this case is as follows. 

[0068] In the first area, the first multiplier circuit 
(MUL1) 18 multiplies data output from the A/D converter 
7 by horizontal correction data H1(i) corresponding to 
that horizontal address L In this fashion, the horizontal 
sensitivity distribution is corrected (first-stage correc- 
tion). Since the central portion of the first area has 
higher sensitivity, a data sequence which includes 
smaller correction coefficients in correspondence with a 
higher sensitivity portion of the image sensing element 
5 than those of other portions, as indicated by the graph 
502 in Fig. 5. 

[0069] The second multiplier circuit (MUL2) 1 9 then 
multiplies the output from the first multiplier circuit 
(MUL1) 18 by vertical correction data V1(j) correspond- 
ing to vertical address j so as to correct the vertical sen- 
sitivity distribution of the first area (second-stage 
correction). This correction data V1 (j) is supplied from 
the V memory (V1) 14 to the second multiplier circuit 
(MUL2) 19 via the selector 17. The correction data V1 (j) 
corresponds to a data sequence which includes smaller 
correction coefficients in correspondence with a higher 
sensitivity portion of the image sensing element 5 than 
those of other portions, as indicated by the graph 504 in 
Fig. 5. 

[0070] As described above, in this embodiment, 
individual pixels in the first area are multiplied by the 
horizontal linear correction data H1(i) and vertical linear 
correction data V1 (j) to implement local sensitivity cor- 
rection in the first area. 

[0071 ] On the other hand, if it is determined in steps 
30, 31 , and 36 in Fig. 9 that the vertical address corre- 
sponding to data (pixel) currently output from the A/D 
converter 7 is smaller than vp and the horizontal 
address is equal to or larger than Ha, it is determined 
that the data of interest belongs to the second area. In 
this case, the select signals HSEL and VSEL as the out- 
put signals from the address discrimination circuit 1 1 
are respectively set at L and H levels in steps 34 and 35. 
Correction data H1(i) stored in the H memory (H1) 12 is 
supplied to the first multiplier circuit (MUL1) 18 via the 
selector 16, and correction data V2(j) stored in the V 
memory (V2) 15 is supplied to the second multiplier cir- 
cuit (MUL2) 19 via the selector 17. The correction proc- 
ess in this case is as follows. 

[0072] In this example, the second area has a flat 



sensitivity distribution, as indicated by 501 in Fig. 5. 
Therefore, data in the second area undergo horizontal 
correction using data having horizontal addresses equal 
to or larger than Ha, i.e., data corresponding to flat char- 

5 acteristics, of the correction data H1(i) indicated by the 
graph 502 in Fig. 5. That is, as for the second area, the 
first multiplier circuit (MUL1) 18 multiplies data output 
from the A/D converter 7 by horizontal correction data 
H1(i) corresponding to that horizontal address i to 

10 implement horizontal correction (first-stage correction). 
[0073] Also, as described above, since the second 
area has a flat sensitivity distribution, vertical correction 
is done using data having vertical addresses smaller 
than Vp, i.e., data corresponding to flat characteristics, 

15 of correction data V2(j) indicated by the graph 505 in 
Fig. 5. That is, as for the second area, the second mul- 
tiplier circuit (MUL2) 19 multiplies the output of the first- 
stage correction by vertical correction data V2Q) corre- 
sponding to vertical address j (second-stage correc- 

20 tion). 

[0074] In this manner, since the second area uses 
linear correction data V2(j) different from the first area 
as the vertical linear correction data, correction that 
matches the sensitivity distribution of the second area 
25 can be done even when the first area suffers sensitivity 
nonuniformity. 

[0075] If it is determined in steps 30, 31, and 36 in 
Fig. 9 that the vertical address corresponding to data 
(pixel) currently output from the A/D converter 7 is equal 

30 to or larger than vp and the horizontal address is 
smaller than Ha in the discrimination process, it is 
determined that the data of interest belongs to the third 
area. In this case, the select signals HSEL and VSEL as 
the output signals from the address discrimination cir- 

35 cuit 1 1 are respectively set at H and L levels in steps 37 
and 38. Correction data H2(i) stored in the H memory 
(H2) 13 is supplied to the first multiplier circuit (MUL1) 
18 via the selector 16, and correction data V1(j) stored 
in the V memory (V1) 14 is supplied to the second mul- 

40 tiplier circuit (MUL2) 19 via the selector 17. The correc- 
tion process in this case is as follows. 
[0076] In this example, the third area has a flat sen- 
sitivity distribution, as indicated by 501 in Fig. 5. There- 
fore, data in the third area undergo horizontal correction 

45 using data having horizontal addresses smaller than 
Ha, i.e., data corresponding to flat characteristics, of the 
correction data H2(i) indicated by the graph 503 in Fig. 
5. That is, as for the third area, the first multiplier circuit 
(MUL1) 18 multiplies data output from the A/D converter 

so 7 by horizontal correction data H2(i) corresponding to 
that horizontal address i to implement horizontal correc- 
tion (first-stage correction). ^ 

[0077] Also, as described above, since the third 
area has a flat sensitivity distribution, vertical correction 
55 is done using data having vertical addresses equal to or 
larger than Vp, i.e., data corresponding to flat character- 
istics, of correction data V1 (j) indicated by the graph 504 
in Fig. 5. That is, as for the third area, the second multi- 
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plier circuit (MUL2) 19 multiplies the output of the first- 
stage correction by vertical correction data V1 (j) corre- 
sponding to vertical address j (second-stage correc- 
tion). 

[0078] To summarize, since the third area uses lin- 5 
ear correction data H2(i) different from the first area as 
the horizontal linear correction data, correction that 
matches the sensitivity distribution of the third area can 
be done even when the first area suffers sensitivity non- 
uniformity, w 
[0079] Furthermore, if it is determined in steps 30. 
31, and 36 in Fig. 9 that the vertical address corre- 
sponding to data (pixel) currently output from the A/D 
converter 7 is equal to or larger than Vp and the hori- 
zontal address is equal to or larger than Ha in the dis- 15 
crimination process, it is determined that the data of 
interest belongs to the fourth area. In this case, the 
select signals HSEL and VSEL as the output signals 
from the address discrimination circuit 11 are respec- 
tively set at H and H levels in steps 39 and 40. The cor- 20 
rection process in this case is as follows. 
[0080] In this example, the fourth area has a central 
higher sensitivity portion, as indicated by 501 in Fig. 5. 
Therefore, the fourth area uses data having horizontal 
addresses equal to or higher than Ha, i.e., data includ- 25 
ing smaller correction coefficients in correspondence 
with a higher sensitivity portion than those of other por- 
tions, of the correction data H2(i) indicated by the graph 
503 in Fig. 5. That is, as for the fourth area, the first mul- 
tiplier circuit (MUL1) 18 multiplies data output from the 30 
A/D converter 7 by horizontal correction data H2(i) cor- 
responding to that horizontal address i to implement 
horizontal correction (first-stage correction). 
[0081] Also, as described above, since the fourth 
area has a central higher sensitivity portion, data having 35 
vertical addresses smaller than Vp, i.e., data including 
smaller correction coefficients in correspondence with a 
higher sensitivity portion than those of other portions, of 
correction data V2(j) indicated by the graph 505 in Fig. 
5. That is, as for the fourth area, the second multiplier 40 
circuit (MUL2) 19 multiplies the output of the first-stage 
correction by vertical correction data V2(j) correspond- 
ing to vertical address j (second-stage correction). 
[0082] In this manner, since the fourth area uses 
linear correction data H2(i) different from the first and 45 
second areas in the horizontal direction, and linear cor- 
rection data V2(j) different from the first and third areas 
in the vertical direction, correction that matches the sen- 
sitivity distribution of the fourth area can be done. 
[0083] Local sensitivity nonuniformity of the image so 
sensing element resulting from variations of color filters 
and the like produced in the manufacturing process is 
highly likely to occur at a plurality of portions in the 
effective area of the image sensing element. Such prob- 
lem becomes more serious with increasing area of the 55 
image sensing element. In order to improve the yield of 
the image sensing element, it is important to correct 
such sensitivity nonuniformity. 



[0084] Hence, like in this embodiment, since at 
least two different types of horizontal linear correction 
data are prepared, at least two different types of vertical 
linear correction data are prepared, and those data are 
selectively used, local sensitivity nonuniformity which is 
likely to occur in a plurality of areas of the image sens- 
ing element can be corrected without preparing correc- 
tion data for one frame. 

[0085] In this embodiment, two addresses Ha and 
Vp are used as addresses used upon switching correc- 
tion data. However, the number of addresses may be 
increased. 

[0086] For example, the horizontal address as the 
boundary between the first and second areas may be 
set at Ha1, and the horizontal address as the boundary 
between the third and fourth areas may be set at Ha2. 
Likewise, the vertical address as the boundary between 
the first and third areas may be set at Vp1, and the ver- 
tical address as the boundary between the second and 
fourth areas may be set at Vp2. 

[0087] Alternatively, the effective area of the image 
sensing element 5 may be segmented into smaller par- 
tial areas (e.g., a total of nine areas; three in the hori- 
zontal direction and three in vertical direction), and 
appropriate linear correction data may be assigned to 
those partial areas. 

[0088] To restate, according to this embodiment, 
the sensitive area of the image sensing element is seg- 
mented into a plurality of areas, and the output from the 
image sensing element is corrected in units of areas by 
combining one of at least two horizontal linear correc- 
tion data, and one of at least two vertical linear correc- 
tion data. In this way, even when the image sensing 
element suffers a plurality of local sensitivity nonuni- 
formity portions resulting from, e.g., sensitivity nonuni- 
formity of color filters, appropriate sensitivity correction 
can be done without individually preparing correction 
data for all pixels. 

[Second Embodiment] 

[0089] The second embodiment of the present 
invention will be described below with reference to Figs. 
6, 1 0, and 1 1 . Note that items that are not touched upon 
in this embodiment invoke the first embodiment. 
[0090] When the image sensing element 5 has a 
device structure (section) indicated by 601 in Fig. 6, the 
incident angle the incoming light makes with the image 
sensing element 5 becomes larger as the pixel position 
becomes closer to the peripheral portion of the image 
sensing element 5. Therefore, neighboring pixels have 
different incoming light amounts. This phenomenon 
becomes more conspicuous as the pixel position sepa- 
rates farther from the optical axis, and is observed in 
both the horizontal and vertical directions. 
[0091] In a direction parallel to the direction of the 
section of 601, the sensitivity (output) of a pixel group 
located at odd positions counted from the edge, and the 
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sensitivity (output) of a pixel group located at even posi- 
tions have opposite characteristics, as indicated by 602 
and 603 in Fig. 6. If the structure indicated by 601 is 
formed in both the horizontal and vertical directions, the 
characteristics at odd. and even positions are different in 
both the horizontal and vertical directions. 
[0092] Therefore, in this embodiment, in order to 
correct such characteristics, correction characteristics 
for the output data from the image sensing element 5 
are changed depending on whether the position 
(address) of the pixel of interest is an odd or even posi- 
tion in the horizontal direction, and an odd or even posi- 
tion in the vertical direction. 

[0093] Fig. 1 0 is a flow chart showing the process in 
the address discrimination circuit 1 1 shown in Fig. 1 . 
[0094] In steps 45, 46, and 51 , the address discrim- 
ination circuit 11 executes a discrimination process on 
the basis of the horizontal and vertical addresses output 
from the address generation circuit 10 in correspond- 
ence with data (pixel) output from the A/D converter 7. If 
vertical address j of data (pixel) currently output from 
the A/D converter 7 is an odd number (i.e., an odd row 
in the matrix of the two-dimensional image sensing ele- 
ment 5) and horizontal address i is an odd number (i.e., 
an odd column in the matrix of the two-dimensional 
image sensing element 5), the flow advances to step 
47. 

[0095] In step 47, the address discrimination circuit 
1 1 sets the output HSEL at L level. As a result, the data 
sequence H1(i) in the H memory (H1) 12 is supplied to 
the first multiplier circuit (MUL1) 18 via the selector 16. 
In step 48, the address discrimination circuit 1 1 sets the 
output VSEL at L level. As a result, the data sequence 
V1(j) in the V memory (V1) 14 is supplied to the second 
multiplier circuit (MUL2) 19 via the selector 17. 
[0096] This state will be explained below with refer- 
ence to Fig. 11. Note that 1101 depicts the two-dimen- 
sional matrix of light-receiving elements in the image 
sensing element 5. Assuming that the pixel of interest at 
that time is a G pixel 1 101a at the upper left corner in 
Fig. 11, i.e., if both horizontal and vertical addresses i 
and j are odd numbers, the first multiplier circuit (MUL1) 
18 corrects data of the pixel 1101a in accordance with 
the horizontal correction data H1(i) indicated by the 
solid curve of a graph 1 102 in Fig. 11. Also, the second 
multiplier circuit (MUL2) 19 further corrects the data of 
the pixel 1 101a in accordance with the characteristics of 
the vertical correction data V1(j) indicated by the solid 
curve of a graph 1 1 04 in Fig. 1 1 . 
[0097] Assume that the pixel of interest is a pixel (R) 
1101b that neighbors the pixel 1101a. In this case, 
since horizontal address i is an even number and verti- 
cal address j is an odd number, the address discrimina- 
tion circuit 1 1 sets the output HSEL at H level in step 49. 
As a result, the data sequence H2(i) in the H memory 
(H2) 13 is supplied to the first multiplier circuit (MUL1) 
18 via the selector 16. In step 50, the address discrimi- 
nation circuit 11 sets the output VSEL at L level. As a 



result, the data sequence V1(j) in the V memory (V1) is 
supplied to the multiplier circuit (MUL2) via the selector 
17. 

[0098] This state will be explained below with refer- 
5 encetoFig. 1 1. The first multiplier circuit (MUL1) 18cor- 
rects the pixel 1101b in accordance with the 
characteristics of horizontal correction data H2(i) indi- 
cated by the dotted curve of a graph 1102 in Fig. 11. 
The second multiplier circuit (MUL2) 19 corrects the 
w pixel 1101b in accordance with the characteristics of 
vertical correction data V1fl) indicated by the solid curve 
of a graph 1105 in Fig. 11. 

[0099] In this manner, correction data are selected 
for pixels G (1101a) —» R (1101b) G —» R —> ... in the 
15 uppermost line of the matrix 1101 in Fig. 11 to imple- 
ment correction. 

[0100] In the second uppermost line of the matrix 
1101 in Fig. 11, since a leftmost pixel (B) 1101c has 
even vertical address j and odd horizontal address i, the 

20 address discrimination circuit 1 1 sets HSEL at L level in 
step 52. As a result, the data sequence H1(i) in the H 
memory (H1) 12 is supplied to the first multiplier circuit 
(MUL1) 18 Via the selector 16. In step 53, the address 
discrimination circuit 11 sets VSEL at H level. As a 

25 result, the data sequence V2(j) in V memory (V2) 15 is 
supplied to the second multiplier circuit (MUL2) 19 via 
the selector 17. 

[0101] This state will be explained below with refer- 
ence to Fig. 1 1. The first multiplier circuit (MUL1) 18 cor- 
30 rects the pixel 1101c in accordance with the 
characteristics of horizontal correction data H1(i) indi- 
cated by the solid curve of a graph 1 103 in Fig. 1 1 . The 
second multiplier circuit (MUL2) 19 further corrects the 
pixel 1101c in accordance with the characteristics of the 
35 vertical correction data V2(j) indicated by the dotted 
curve in a graph 1 1 04 in Fig. 1 1 . 
[0102] Assume that the pixel of interest is a pixel 
(G) 1101d that neighbors the pixel 1101c. In this case, 
since both horizontal and vertical addresses i and j are 
40 even numbers, the address discrimination circuit 11 
sets HSEL at H level in step 54. As a result, the data 
sequence H2(i) in the H memory (H2) 13 is supplied to 
the first multiplier circuit (MUL1) via the selector 16. In 
step 55, the address discrimination circuit 1 1 sets VSEL 
45 at H level. As a result, the data sequence V2fl) in the V 
memory (V2) 15 is supplied to the second multiplier cir- 
cuit (MUL2) 19 via the selector 17. 
[01 03] This state will be explained below with refer- 
ence to Fig. 1 1 . The first multiplier circuit (MUL1 ) 1 8 cor- 
so rects the pixel 1101d in accordance with the 
characteristics of horizontal correction data H2(i) indi- 
cated by the dotted curve of the graph 1103 in Fig. 11. 
The second multiplier circuit (MUL2) 19 further corrects 
the pixel 1 101 d in accordance with the characteristics of 
55 the vertical correction data V2(j) indicated by the dotted 
curve in the graph 1 105 in Fig. 11. 
[0104] In this manner, correction data are selected 
for pixels B (1101c) -> G (1101d) — >B— >G-»B-»in 
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the second uppermost line of the matrix 1101 in Fig. 1 1 
to implement correction. 

[0105] As described above, according to this 
embodiment, since horizontal correction data is 
switched depending on whether the horizontal address 5 
is an even or odd value, and vertical correction data is 
switched depending on whether the vertical address is 
an even or odd value, color shading due to different 
incoming light amounts on neighboring pixels in the 
image sensing element with the device structure indi- 10 
cated by 601 in Fig. 6 can be prevented. 
[0106] In the above-mentioned embodiment, two 
different types of linear correction data are used in each 
of the horizontal and vertical directions. However, the 
present invention is not limited to this. For example, one 15 
or a plurality of correction data may be used in units of 
R, G, and B. Alternatively, when complementary color 
filters are used as color filters, for example, four different 
types of correction data may be used in each of the hor- 
izontal and vertical directions. 20 
[01 07] According to this embodiment, each horizon- 
tal line is grouped (into, e.g., even and odd lines) and 
each vertical line is grouped (into, e.g., even and odd 
lines) in accordance with the device structure of the 
image sensing element and the layout of the photo- 25 
graphing optical system, and different linear correction 
data are applied to these groups to correct the output 
from the image sensing element. In this manner, color 
shading or the like produced by a combination of the 
device structure of the image sensing element and the 30 
photographing optical system can be corrected. 

[Third Embodiment] 

[0108] The third embodiment of the present inven- 35 
tion will be described below with reference to Figs. 12, 
13, and 14. 

[0109] This embodiment relates to a correction 
method that accurately removes both white shading 
produced by a combination of a main photographing 40 
optical system and microlenses on an image sensing 
element, and sensitivity nonuniformity produced by var- 
iations of color filters on the image sensing element in 
the manufacture process. 

[0110] Fig. 12 is a block diagram showing the 45 
arrangement of a camera (image sensing device) 
according to the third embodiment of the present inven- 
tion. This camera has an arrangement to which an HV 
memory 20 and third multiplier circuit (MUL3) 21 are 
added to that of the camera shown in Fig. 1 , and from so 
which the H memory (H2) 13, V memory (V2) 15, 
address discrimination circuit 11, and selectors 16 and 
17 are omitted. 

[0111] The HV memory 20 stores R, G, and B cor- 
rection data in units of blocks, which are defined for the 55 
two-dimensional matrix of the image sensing element 5 
in units of predetermined areas each consisting of a plu- 
rality of pixels, as shown in Fig. 13. For example, a filter 



matrix on the left side in Fig. 13 indicates a so-called 
Bayer matrix, and each block bounded by the dotted line 
include eight G components and four each R/B compo- 
nents, and sensitivity correction data for this portion use 
identical values in units of colors. 
[0112] As indicated by the right side in Fig. 13, as a 
G correction data sequence, correction data corre- 
sponding to block 1, correction data corresponding to 
block Z,..., correction data corresponding to last block n 
are successively stored in the H V memory 20. An R/B 
correction data sequence has a similar data sequence. 
Therefore, local sensitivity nonuniformity resulting from 
color filters and the like can be corrected using the HV 
memory 20 having a memory size smaller than correc- 
tion data for all pixels of the image sensing element 5. 
[0113] In general, the characteristics of sensitivity 
nonuniformity produced by variations of color filters or 
the like in the manufacture process remain the same 
even when the optical conditions or the like of the pho- 
tographing optical system have changed. However, 
since the characteristics of the aforementioned white 
shading components change when the focal length 
(strictly, pupil position) of the photographing optical sys- 
tem has changed, the value must be re-set. 
[0114] The correction process in this embodiment 
will be explained below with reference to the flow chart 
in Fig. 14. 

[0115] In step 60, correction data to be set in the H 
memory (H1 ) 1 2 is read out from the internal nonvolatile 
memory of the overall control CPU 1, and is stored in 
the H memory (H1) 12. In step 61, correction data to be 
set in the V memory (V1) 14 is read out from the internal 
nonvolatile memory of the overall control CPU 1 , and is 
stored in the V memory (V1) 14. Note that the correction 
data to be set in the H memory (H1) 12 and V memory 
(V1) 14 are the same as the linear correction data 
explained in the first embodiment, i.e., are determined 
in accordance with the characteristics of the image 
sensing element 5 measured during the manufacturing 
process, and are pre-stored in the internal nonvolatile 
memory of the overall control CPU 1 . 
[0116] In step 62, the aforementioned correction 
data in units of blocks 1 to n of the image sensing ele- 
ment are read out from the internal nonvolatile memory 
of the overall control CPU 1 and are stored in the HV 
memory 20. These data are also determined in accord- 
ance with the characteristics of the image sensing ele- 
ment 5 measured during the manufacturing process, 
and are pre-stored in the internal nonvolatile memory of 
the overall control CPU 1 . 

[0117] With these setups, the first multiplier circuit 
(MUL1) 18 multiplies the output from the A/D converter 
7 by the value from the H memory (hi) 12 output by 
designating horizontal address i by the address genera- 
tion circuit 10. Then, the second multiplier circuit 
(MUL2) 19 multiplies the output from the first multiplier 
circuit (MUL1) 18 by the value of the V memory (V1) 14 
output by designating vertical address j by the address 
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generation circuit 10. With the two operations, the influ- 
ences of white shading or the like produced by a combi- 
nation of the photographing lens 4 and the microlenses 
of the image sensing element 5 can be corrected. 
[0118] Furthermore, the third multiplier circuit 5 
(MUL3) 21 multiplies the output from the second multi- 
plier circuit (MUL2) 19 by the value of the HV memory 
20 output by designating the horizontal and vertical 
addresses by the address generation circuit 10. The HV 
memory 20 stores correction data corresponding to the 10 
individual blocks of the two-dimensional matrix of the 
image sensing element 5, as described above. With the 
correction data, local sensitivity non uniformity of color 
filters of the image sensing element 5 can be corrected. 
[0119] It is checked in step 63 if a field angle setting 15 
unit 30 shown in Fig. 12 (e.g., a zoom operation button 
or the like) is operated by the operator. If NO in step 63, 
this step repeats itself; if YES in step 63, the flow 
advances to step 64. 

[0120] In step 64, the focal length information value 20 
output from the .focal length information detector 2 is 
read. The overall control CPU 1 re-computes correction 
data to be stored in the H memory (H1) 12 and that to 
be stored in the V memory (V1) 14 in accordance with 
predetermined functions, as shown in, e.g., Fig. 15. 25 
[0121] In Fig. 15, since the incoming light amount at 
the peripheral portion of the frame decreases with 
decreasing focal length, correction coefficients (gains) 
for the peripheral portion are set to be larger than those 
for the central portion. On the other hand, since the 30 
incoming light amount at the peripheral portion of the 
frame changes little with increasing focal length, the dif- 
ference between the correction coefficients for the 
peripheral portion and those for the central portion is set 
to be small. 35 
[0122] Therefore, in steps 65 and 66, new correc- 
tion data are obtained by a method of multiplying the 
correction data sequence by a coefficient (a function 
using the focal length as a variable) that changes with 
respect to the focal length to have a predetermined rela- 40 
tionship, a method of preparing, as a table, correction 
data sequences in correspondence with focal lengths in 
the internal nonvolatile memory or the like of the overall 
control CPU 1, and selecting the correction data 
sequence corresponding to the current focal length from 45 
this table, or the like. The obtained new correction data 
are re-set in the H memory (H1) and V memory (V1), 
and the flow returns to step 63. 

[0123] As described above, in this embodiment, 
sensitivity nonuniformity of the image sensing element, so 
which is determined depending on the state of the pho- 
tographing optical system, is corrected using the hori- 
zontal and vertical linear data sequences, while 
sensitivity nonuniformity or the like of color filters pro- 
duced at the time of manufacture of the image sensing 55 
element is corrected using the correction data 
sequence assigned units of blocks each consisting of a 
plurality of pixels. Also, components which are deter- 
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mined depending on the state of the photographing opti- 
cal system are corrected by re-setting correction data in 
correspondence with the state of the optical system 
when the state (e.g., the focal length, pupil position, 
aperture, or the like) of the photographing optical sys- 
tem has changed. 

[01 24] According to this embodiment, since the out- 
put from the image sensing element is corrected by 
combining linear correction data in the horizontal and 
vertical directions of the image sensing element, and 
correction data determined in units of blocks each con- 
sisting of a plurality of pixels in the two-dimensional 
directions of the image sensing element, both white 
shading mainly produced due to the influence of the 
optical system, and local sensitivity nonuniformity of, 
e.g., color filters produced in the manufacture process 
of the image sensing element can be corrected. 

[Other Embodiments] 

[0125] Note that the present invention may be 
applied to either a system constituted by a plurality of 
devices, or an apparatus consisting of a single equip- 
ment. 

[01 26] The objects of the present invention are also 
achieved by supplying a storage medium (or recording 
medium), which records a program code of a software 
program that can implement the functions of the above- 
mentioned embodiments to the system or apparatus, 
and reading out and executing the program code stored 
in the storage medium by a computer (or a CPU or 
MPU) of the system or apparatus. In this case, the pro- 
gram code itself read out from the storage medium 
implements the functions of the above-mentioned 
embodiments, and the storage medium which stores 
the program code constitutes the present invention. The 
functions of the above-mentioned embodiments may be 
implemented not only by executing the readout program 
code by the computer but also by some or all of actual 
processing operations executed by an operating system 
(OS) running on the computer on the basis of an 
instruction of the program code. 

[0127] Furthermore, the functions of the above- 
mentioned embodiments may be implemented by some 
or all of actual processing operations executed by a 
CPU or the like arranged in a function extension board 
or a function extension unit, which is inserted in or con- 
nected to the computer, after the program code read out 
from the storage medium is written in a memory of the 
extension board or unit. 

[01 28] According to the present invention, for exam- 
ple, sensitivity nonuniformity of the image sensing ele- 
ment can be satisfactorily corrected. 
[01 29] As many apparently widely different embodi- 
ments of the present invention can be made without 
departing from the spirit and scope thereof, it is to be 
understood that the invention is not limited to the spe- 
cific embodiments thereof except as defined in the 
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claims. 

[0130] An image sensing device of this invention 
corrects sensitivity non uniformity of an image sensing 
element. This image sensing device has an image 
sensing element (5) having a two-dimensional matrix of 5 
pixels, a first multiplier circuit (18) for multiplying the out- 
put from the image sensing element (5) output via an 
A/D converter (7) by horizontal linear correction data, a 
second multiplier circuit (19) for multiplying the product 
of the first multiplier circuit by vertical linear correction 10 
data, a selector (16) for selecting horizontal linear cor- 
rection data, and a selector (17) for selecting vertical lin- 
ear correction data. The device switches the selectors 
(16, 17) in accordance with the position (horizontal and 
vertical addresses) of the pixel of interest. 15 

Claims 

1. An image sensing device comprising: 

20 

an image sensing element having a two-dimen- 
sional matrix of pixels; 

first multiplier means for multiplying a row of an 
image sensed by said image sensing element 
by horizontal linear correction data; 25 
second multiplier means for multiplying a col- 
umn of the image by vertical linear correction 
data; and 

change means for changing at least one of the 
horizontal and vertical linear correction data in 30 
accordance with a position of a pixel of interest 
in the image, 

wherein said first and second multiplier means 
correct a value of each pixel of the image. 

35 

2. The device according to claim 1 , wherein the image 
is segmented into a plurality of partial areas, and 
said change means changes at least one of the hor- 
izontal and vertical linear correction data in accord- 
ance with the partial area to which the pixel of 40 
interest belongs. 

3. The device according to claim 1 , wherein the image 
is segmented into a plurality of partial areas, and 
said change means changes both the horizontal 45 
and vertical linear correction data in accordance 
with the partial area to which the pixel of interest 
belongs. 

4. The device according to claim 1, wherein said 50 
change means has two different types of horizontal 
linear correction data, alternately switches the hori- 
zontal linear correction data in units of rows, and 
supplies the selected data to multiplication by said 
first multiplier means. 55 

5. The device according to claim 4, wherein said 
change means has two different types of vertical 



linear correction data, alternately switches the ver- 
tical linear correction data in units of columns, and 
supplies the selected data to multiplication by said . 
second multiplier means. 

6. The device according to claim 5, wherein said 
image sensing element includes color filters in a 
Bayer matrix. 

7. The device according to claim 1 , wherein the image 
is segmented into a plurality of groups each of 
which is defined by an area as a two-dimensional 
matrix of a plurality of pixels, 

said device further comprises third multiplier 
means for multiplying each group by two- 
dimensional correction data given by a row and 
column, and said change means includes 
means for changing the two-dimensional cor- 
rection data in accordance with the group to 
which the pixel of interest in the image belongs. 

8. The device according to claim 1, wherein said 
change means includes means for changing at 
least one of the horizontal and vertical linear cor- 
rection data in accordance with a state of a photo- 
graphing optical system. 

9. The device according to claim 1, wherein said 
change means includes means for changing at 
least one of the horizontal and vertical linear cor- 
rection data in accordance with one of a focal 
length, field angle, and aperture of a photographing 
optical system or a combination thereof. 

10. An image processing apparatus for processing an 
image supplied from an image sensing element 
having a two-dimensional matrix of pixels, compris- 
ing: 

first multiplier means for multiplying a row of an 
image sensed by said image sensing element 
by horizontal linear correction data; 
second multiplier means for multiplying a col- 
umn of the image by vertical linear correction 
data; and 

change means for changing at least one of the 
horizontal and vertical linear correction data in 
accordance with a position of a pixel of interest 
in the image, 

wherein said first and second multiplier means 
correct a value of each pixel of the image. 

1 1 . An image processing method comprising: 

the first multiplier step of multiplying a row of an 
image sensed by an image sensing element 
having a two-dimensional matrix of pixels by 
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horizontal linear correction data; 
the second multiplier step of multiplying a col- 
umn of the image by vertical linear correction 
. data; and 

the change step of changing at least one of the 5 
horizontal and vertical linear correction data in 
accordance with a position of a pixel of interest 
in the image, 

wherein a value of each pixel of the image is 
corrected in the first and second multiplier 10 
steps. 

A memory medium storing an image processing 
program, said program including: 

15 

the first multiplier step of multiplying a row of an 
image sensed by an image sensing element 
having a two-dimensional matrix of pixels by 
horizontal linear correction data; 
the second multiplier step of multiplying a col- 20 
umn of the image by vertical linear correction 
data; and 

the change step of changing at least one of the 
horizontal and vertical linear correction data in 
accordance with a position of a pixel of interest 25 
in the image, 

wherein a value of each pixel of the image is 
corrected in the first and second multiplier 
steps. 

30 



40 



45 



50 



55 
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FIG. 8 
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